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Abstract 

This paper will provide a methodology and progress report from a multivocal thematic 
synthesis being conducted on an extensive, diverse body of empirical studies. The study data 
includes a corpus of peer-reviewed empirical literature sharing a common reference published in 
English between 2000 and 2014. 

In this study, data to be synthesized share a common reference to the Garrison, Anderson, 
and Archer (2000) seminal paper introducing the Community of Inquiry framework, one of the 
most influential distance and blended learning theories published (Halverson, Graham, Spring, 
Drysdale, & Henrie, 2013). The study has two deliverables: 1) a multivocal thematic synthesis 
analysing contexts, populations, learning and teaching environments, and aspects of learning 
being studying through the lens of the Col framework; and 2) an online, open, dynamic 
compendium of all Col empirical research citing the Garrison et al., 2000, seminal paper. The 
purpose of the study is two-fold: to bring systematization and organization to a large body of 
research to support the work of other researchers, practitioners, and policy makers; and to 
provide an up-to-date methodology framework for creating an online database from large bodies 
of literature with a common reference. 

The data used in the thematic synthesis study consisted of full-text copies of peer- 
reviewed research articles. The data was analysed using an open, constant-comparative, three- 
stage thematic synthesis methodology. As the procedures used to gather, organize, and archive 
the data in this study could be applied to any multivocal body of research, the methodological 
processes was documented in detail to provide a framework for future research and may serve as 
an exemplar for sharing research data as an open-access resource. 
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Resume 

Cet article offrira une methodologie et un rapport sur la progression d’une synthese 
thematique multivocale en cours de realisation sur un vaste ensemble d’etudes empiriques 
variees. Les donnees d’etude comprennent un corpus de documents empiriques evalues par les 
pairs ayant une reference commune et publies en anglais de 2000 a 2014. 

Dans cette etude, les donnees a etre synthetisees ont en commun une reference a 1’article 
phare de Garrison, Anderson et Archer (2000) ayant introduit le cadre de la communaute 
d’apprentissage, l’une des theories les plus influentes publiees sur l’apprentissage a distance et 
l’apprentissage mixte (Halverson, Graham, Spring, Drysdale et Henrie, 2013). L’etude vise deux 
realisations : 1) une synthese thematique multivocale analysant les contextes, les populations, les 
environnements d’apprentissage et d’enseignement, ainsi que les aspects de l’apprentissage 
etudies dans l’optique du cadre de la communaute d’apprentissage; et (2) une collection en ligne, 
ouverte et dynamique, de toutes les etudes empiriques sur la communaute d’apprentissage citant 
l’article fondamental de Garrison et collegues (2000). L’etude a deux raisons d’etre : 
systematiser et organiser un vaste ensemble de documents de recherche pour appuyer le travail 
d’autres chercheurs, praticiens et decideurs, et foumir un cadre methodologique mis a jour pour 
la creation d’une base de donnees en ligne a partir de vastes ensembles de documents ayant une 
reference commune. 

Les donnees utilisees dans la synthese thematique sont formees des textes integraux des 
articles de recherche evalues par les pairs. Les donnees ont ete analysees a l’aide d’une methode 
comparative constante et ouverte de synthese thematique en trois phases. Comme les procedures 
utilisees pour recueillir, organiser et archiver les donnees dans cette etude pourraient etre 
appliquees a tout ensemble de documentation multivocale, les processus methodologiques ont ete 
documentes en detail afin de fournir un cadre pour des recherches futures et peuvent servir 
d’exemple pour le partage de donnees de recherche comme ressource ouverte. 


Introduction 

The project described in this paper arises from a doctoral dissertation researching the 
influence of the Community of Inquiry (Col) framework (Garrison, Anderson, & Archer, 2000) 
in distance and online education. Common to many academic dissertation undertakings, the 
purpose of the current project is to bring organization to, and to make sense of previous studies 
and publications in a specific topic or field; to delineate what is known or not known about the 
topic; and to identify a specific area of research within the topic that both interests the researcher 
and fulfills the expectation that the research will add important information to the body of 
knowledge on the topic. 

For many research projects, the preceding steps of reviewing existing knowledge and 
identifying areas that make a case for further research are followed by steps that involve design 
and implementation of an appropriate research strategy to gather data, analyze data, describe 
findings, and then write up of the conclusions. However, the research project described in this 
paper diverges somewhat from the traditional research project pattern described above as the 
data being gathered and analyzed consists of empirical studies conducted by others. The use of 
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previously published research materials as data places this project in the “literature review” 
article category of the American Psychological Association (APA) and fits within the description 
of articles “including research synthesis and meta-analyses, [which] are citical evaluations of 
material that has already been published” (American Psychological Association [APA], 2010, p. 
10 ). 


The body of literature chosen for this project shares a common reference to a seminal 
paper published by Garrison, Anderson, and Archer (2000). Since the Community of Inquiry 
framework was posited by Garrison et al. in 2000, hundreds of researchers (Garrison, Anderson, 
& Archer, 2010; Garrison & Arbaugh, 2007; Halverson, Graham, Spring, & Drysdale, 2012; 
Rourke & Kanuka, 2009) have referenced the framework and its three elements of social 
presence, cognitive presence, and teaching presence as the basis for empirical studies of distance, 
blended and online learning, teaching, student success and retention, as well as course and 
program design. Few education-focused academic publications have generated this amount of 
interest or empirical action (Halverson, Graham, Spring, Drysdale, & Henrie, 2013). 
Chronological categorization of Google Scholar© citations of the seminal Col article provides 
evidence that the model continues to resonate within the education community despite 
extraordinary distance education and technological innovation, and copious distance education 
research in the intervening 14 years (See Appendix A). 

The corpus of empirical research on the Col model is substantial. Despite the extent of 
empirical Col research, an up-to-date thematic research synthesis of the Col model or an easily 
accessible database of this body of research literature does not exist. The proliferation of 
journals, online databases and archives and search engines such as Google Scholar© do facilitate 
literature searches (Fink, 2013) but there exists no easy way to organize and assimilate hundreds 
or thousands of articles on a particular topic. The absence of a Col research synthesis or 
compendium means that researchers interested in the Col model must seek and construct this 
literary resource individually and probably repetitively - a daunting and time-consuming task 
considering the quantity and diversity of Col empirical research. As the body of Col research 
continues to grow by several hundred articles each year, collecting and making sense of the 
accumulated knowledge becomes an even more formidable task, particularly for novice 
researchers. 

In 1976, in his book introducing meta-analysis, Glass observed “Our problem is to find 
the knowledge in the information. We need methods for the orderly summarization of studies so 
that knowledge can be extracted from the myriad individual researches” (p. 4). The preceding 
statement aligns closely with the problem that this study is based upon, namely how to make 
sense of a diverse body of research reflecting different purposes, perspectives and contents but 
sharing a common foundation of the Community of Inquiry framework. Ogawa and Malen 
(1991) confronted this problem by devising a pragmatic research method they termed an 
exploratory case study method for multivocal literatures. They define multivocal literature as 
being comprised of all accessible writings on a common topic embodying diverse views or 
voices, appearing in a variety of forms and reflecting different purposes, perspectives and 
informational basis, addressing different aspects of the topic and incorporating different research 
or non-research logics. 
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Ogawa and Malen describe exploratory case study as “well-suited to the examination of 
complex contemporary phenomena.. .because the distinctive features of the exploratory case 
study parallel the major characteristics of multivocal literatures” (p. 274). This present study 
integrates multivocal exploratory concepts that include rigorous examination and recording of 
document characteristics; organizing, categorizing and open coding in order to elicit themes and 
patterns; and creation of a “formal, retrievable data base” (p. 280). 

Deliverables 

The study will produce two deliverables: (a) a multivocal thematic synthesis analysing 
contexts, populations, learning and teaching environments, and aspects of learning being 
studying through the lens of the Col framework, and (b) an online, open, dynamic compendium 
of all Col empirical research citing the Garrison et al. (2000) seminal paper. The purpose of the 
study is two-fold: to bring systematization and organization to a large body of research to 
support the work of other researchers, practitioners, and policy makers; and to provide an up-to- 
date methodology framework for creating an online, digital database from large bodies of 
literature with a common reference. 

A thematic synthesis of empirical Col research for the wider community of distance 
education researchers, practitioners, administrators, and instructional designers may facilitate 
transference of empirical research to applied practice. Of the 12 synthesis methods discussed by 
Bamett-Page and Thomas (2009), thematic synthesis is identified as one of the methods most 
likely to “directly inform policy and practice” (p. 6). For the distance education research 
community, the synthesis may also identify areas of abundant or sparse research, dissention or 
agreement on aspects of the model, as well as innovative and pragmatic applications. 

Upon completion of the project, the online archive will contain abstracts and publication 
information for all Col empirical research articles that cite the Garrison et al. (2000) seminal 
paper. The archive will be structured such that users will be able to sort, search, and query the 
collection for their own research or synthesis purposes. In keeping with copyright stipulations for 
fair dealing for academic purposes, Canadian Copyright Act (Copyright Act, 2001) full-text 
documents will be used by the researcher for research purposes only and not added to the online 
archive unless allowed by copyright. 

Method 

For the pilot study, Google Scholar© was used to identify all documents citing Garrison, 
Anderson and Archer (2000); Zotero ™ research management software was then used to 
facilitate downloading, saving, and tagging the first 50 documents from this citation list as a 
calibration and methodological refinement exercise. Zotero™ automatically creates academic 
citations, downloads, and attaches full-text files (where accessible), saving substantial amounts 
of time both in acquiring and cataloguing documents particularly when studies are published in 
open journals. Full-text copies of all 50 documents were obtained through online digital 
databases or institutional libraries. As all study dates were 2000 or later, 49 (98%) of the first 50 
full-text versions were readily available in digital format with meta-data which also facilitated 
document examination for key study parameters making it possible to retrieve and code 
documents rapidly. 
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The abstract, body, and reference sections of each document were examined to determine 
whether the paper contained a valid citation of the Garrison et al., (2000) seminal paper. A pro¬ 
forma tool, developed specifically for this project to record key document categorization 
information, was created for each study artefact (See Appendix B). The pro-forma tool is an 
original design, but is loosely based on examples provided by the Campbell Collaboration 
(Hammerstrpm, Wade, & Jprgensen, 2010). While the pro-forma tool was invaluable in ensuring 
consistency in document acquisition, identification, and categorization to safeguard data 
collection integrity and facilitate coding, it is not to be considered an a priori code table. The 
pro-forma tool has been piloted, and reviewed by colleagues and has proven to be an essential 
part of the data collection, coding, and database methodology used in this project. 

As of March 19, 2014, Google Scholar© listed 1,902 citations of the Garrison et al., 
(2000) paper. The completed study will eventually include all documents citing the seminal 
paper, but this present pilot phase of the project was limited to the first 50 most relevant 
documents as identified by Google Scholar©. Identical document citation search parameters 
were applied using Harzing’s (2014) open-source citation analysis tool, Publish or Perish© 
version 4.6.3, and returned the same citation count. Approximately 50 percent of the papers 
examined so far contained only one very brief reference to Garrison et al., (2000). Studies 
containing a sole citation of the seminal paper will not be included in the project analysis phase, 
but they will, however, be tagged accordingly and listed in the final project database to inform 
other researchers. 

The populated and tagged Zotero ™ database was then exported to a spreadsheet program 
which was used to analyze document characteristics and timelines. Subsequent phases of this 
project will involve exporting representational full-text research articles from the Zotero™ 
database into ATLAS-ti™ for thematic coding and content analysis. A three-stage free coding 
protocol for thematic synthesis developed by Thomas and Harden (2008) will frame the content 
analysis work. 

Analysis 

Analysis in the full study will take place in two distinct analysis phases as described 
below; however, for this pilot, only phase 1 was completed. 

Analysis Phase 1. Research study characteristics were analyzed quantitatively using data 
recorded on the year of artefact publication, level of evidence, study sample size, geographic 
location, research type, data collection method, data analysis method, education environment 
(online, etc.), educational context of study (grad, undergrad, etc.) participants, study discipline, 
and component of Col component being researched. This analysis was conducted using a 
spreadsheet program. 

Analysis Phase 2. Qualitative thematic content analysis coding will be conducted on at 
least 20 studies from each chronological subsample group. Studies to be included in the thematic 
synthesis will be identified during analysis phase 1 and will include studies that are based 
primarily on the Col framework or components thereof. With each data subset sorted and 


Conducting a Multivocal Thematic Synthesis on an Extensive Body of Literature 


5 



CJLT/RCAT Vol. 42(2) - Special Issue 


grouped chronologically by year, 20 full-text articles will be chosen based on the Google Scholar 
ranked relevance from each chronological group. These 20 artefacts will be subjected to 
paragraph-by-paragraph open-coding based on the research questions, namely: What is the 
nature, focus, and context of empirical research that has been undertaken on the Col framework 
since the publication of the Garrison et al. (2000) seminal paper?; and What factors have 
contributed to the continuing popularity of the Col framework? 

The coding protocol proposed for this dissertation research is similar to the thematic 
synthesis process followed by Thomas and Harden (2008). Each included study will be read and 
re-read using a problematizing approach that “goes beyond the semantic content of the data, and 
starts to identify or examine the underlying ideas, assumptions, conceptualizations, and 
ideologies that are theorized as shaping or informing the semantic content of the data” (Braun & 
Clarke, 2006, p. 84). Attride-Sterling (2001) refers to this initial set of codes as basic themes or 
“lower-order premises evident in the text” (p. 388). 

When all subsample groups have been coded, qualitative data analysis software will be 
used to analyze lower-order codes to discover analytical themes. Finally, analytical themes will 
be analyzed and mapped chronologically, subjectively, and technologically against the Col 
framework itself as well as against developments in distance education pedagogy and 
technology, and major events in the life of the Col model such as book publications, Col 
measurement tool development, and special issues of journals. Analysis by year may reflect use 
of the Col with technology such as chat, audio-, or video-conferencing that didn’t exist when the 
original paper was published, augmentation of asynchronous discussion forums with increased 
visual elements such as still pictures and use of wikis, blogs, and social media. 

As the goal of this study is to gather and report factual knowledge, not to judge or 
critique other studies or interpret data, researcher honesty and meticulousness have more 
influence on the outcomes of the research than potential biases. Further, the choice of thematic 
synthesis as a methodology lessens the influence of bias. Barnett-Page and Thomas (2009) state 
that while thematic synthesis involves some interpretation of data, its synthetic product is more 
likely to be “reproducible and correspond to a shared reality” (p. 6), than most other research 
synthesis methods. 

Results 

Preliminary analysis of the pilot sample of 50 peer-reviewed Col research studies for the 
period 2000 through to 2013 that referenced the seminal Garrison et al. (2000) paper revealed a 
wide range of study types, some based on the entire Community of Inquiry (Col) framework, 
others on individual or combinations of social, teaching, and cognitive presences. The pilot 
dataset included 41 journal articles, 4 conference papers, 3 book sections, 1 book, and 1 thesis. 
The earliest item was published in 1999, the most recent in 2014. If there was only one brief in- 
text citation of the seminal article included as background in the literature review section, the 
item was categorized as “1BC - one brief citation,” tagged and excluded from further 
examination. As a result of this initial document examination, 24 articles were excluded from 
further examination. 

Three main categories of study foci appear to be: (a) validation of the Col model itself 
and of Col measurement instruments (Arbaugh et al, 2008), (b) applicability of the framework 
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and relationships of the three component presences in particular contexts, and (c) use of the Col 
framework as an underpinning framework for course or program design or redesign, and for 
measuring the effect of educational interventions primarily in post-secondary or higher distance, 
in blended or online formal education settings. 

The following tables report quantitative analysis of study sample characteristics: 


Table 1 

Study Level of Evidence 


Level of Evidence 

Studies 

Aggregated 

Participants 

Average 
Sample Size 

LoE 4 - small-sample, single-site qualitative, 
mixed methods or quantitative studies 

14 

1,036 

82 

LoE 3 - comparative, multicourse case studies or 
large- sample quantitative studies category 

7 

3,660 

610 

LoE 5 - descriptive studies and/or self-report 
stories. 

2 

0 

n/a 

LoE 1 - randomized experimental trials or meta¬ 
analyses 

0 

n/a 

n/a 

LoE 6 - opinion of respected authorities or expert 
communities 

0 

n/a 

n/a 


Table 2 

Geographic Location of Study 


Location 

Studies 

% 

United States 

14 

61 

Canada 

4 

17 

United States and other location 

2 

9 

Other (Switzerland, South Pacific) 

3 

13 

or not stated 
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Table 3 

Research Method 


Method 

Studies 

% 

Quantitative 

13 

57 

Mixed Methods 

7 

30 

Qualitative 

2 

9 

Not clearly stated 

1 

4 


Table 4 

Study Data Collection Method 


Data Collection Method 

Studies 

% 

Survey(ies) 

14 

61 

Discussion forum postings 

5 

22 

Interview(s) or focus group(s) 

2 

9 

Data rubric 

1 

4 

Not stated 

1 

4 


Table 5 

Study Data Analysis Method 


Data Analysis Method 

Studies 

% 

Statistical analysis 

12 

52 

Content or text analysis of discussion 

7 

TO 

postings, open-ended questions, text chats 


J U 

Thematic or grounded theory-type 

7 

13 

qualitative analysis 

D 

Participant group consensus 

1 

4 
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Table 6 

Research Population Educational Setting 


Educational Setting 

Studies 

% 

Online 

18 

78 

Blended 

2 

9 

Comparative online to face-to-face 

2 

9 

Not stated 

1 

4 


Table 7 

Research Population Education Level 


Education Level 

Studies 

% 

Graduate 

10 

44 

Undergraduate 

8 

35 

Post-graduate or professional development 

5 

22 

K-12 

0 

0 


Table 8 

Research Population Discipline 


Discipline 

Studies 

% 

Education 

10 

43 

Business 

4 

17 

Computer Science 

1 

4 

Health 

1 

4 

Other (food science, law, English 
as a second language) 

4 

17 

Not stated or mixed 

3 

13 


Practical applications of the framework ranged from chapters in textbooks for online 
teachers (e.g., Palloff & Pratt, 2007) and foundations for teacher education programs (e.g., 
Wilson & Stacey, 2004) to Twitter © and social media integration into classrooms (e.g., Dunlap 
& Lowenthal, 2009). Research settings included blended, (Ayyavoo, 2013), online (Gorsky & 
Blau, 2009), and face-to-face (Zhan & Mei, 2013) education, in elementary (Jimoyiannis & 
Angelaina, 2012), secondary (Dikkers, Whiteside, & Lewis, 2013), post-secondary (Lu & 
Churchill, 2012), and post-graduate (Han & Hill, 2007) education contexts. Research topics were 
diverse; topics from the 50 studies included technology affordances (Alamu, Swan, & Schroeder, 
2013), homeschooling (Thomson, 2010), business education (Cubric, 2013; Daspit & D’Souza, 
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2012; Gupta & Bharadwaj, 2013), curriculum design (Ice, Burgess, Beals, & Staley, 2012; Ke & 
Chavez, 2013), cognitive psychology (Alkali & Amichai-Hamburger, 2004), educational 
technology (Lambert & Fisher, 2009), and learning flow (Tomberg, 2013). 

Challenges associated with accessing this corpus of literature became evident during the 
pilot study of Col empirical research articles. As anticipated, the pilot study revealed a wide 
variety of closed and open publication venues, many requiring subscriptions and passwords, 
complicating the acquisition of articles for inclusion in research literature reviews. However, the 
processes developed, documented, and calibrated during the pilot have provided a viable 
framework for the full study. In particular, the Proforma template form provided very effective 
for systematic study characteristic recording and database entry. Version 1 of the Proforma 
template was revised significantly as a result of the pilot study (See Appendix B). 

The online database produced during the pilot study has been beta tested and some 
revisions to the Zotero tagging procedure based on pilot study results are recommended for the 
full study. This pilot study has been presented to an online research group with encouraging 
results. The pilot database has already proved useful in answering queries from other researchers 
seeking specific Col articles. 


Summary and Conclusion 

Many significant distance education developments and innovations have occurred since 
the Garrison et al. (2000) asynchronous, text-based Community of Inquiry study was completed. 
Some of those developments include widespread adoption of learning management systems 
(LMS) (WebCT 1997, Blackboard™, Desire to Learn™ 1999, Canvass™, epath™ 1999, 
Moodle™ 2002); significantly improved synchronous web conferencing and communication 
media such as Skype™, Macromedia Breeze (now Adobe Connect™) in 2003; improved inter¬ 
compatibility between computer operating systems (Sharable Content Object Reference Model 
or SCORM) in 2004; Connectivism, Facebook™, Web 2.0, social networks in 2005; iPhone™, 
mobile learning in 2007; massive open online courses (MOOCs); and the Android OS in 2008. 
Yet, studies based on the Col framework continue to proliferate. Providing an accessible, 
organized, tagged database of Community of Inquiry research will support the work of future 
researchers, practitioners, and policy makers. Categorization of Google Scholar© citations sorted 
by publication date demonstrate that the Col framework continues to resonate with educational 
researchers and practitioners despite these significant evolutions in distance education 
technology and context (See Appendix A). 

Although the volume of studies to be included in the full study will be prodigious, 
incorporation of software tools such as the Zotero ™ online research tool, availability of digital 
documents with embedded meta-data, online database programs with import and export 
capabilities, and qualitative research analysis software such as ATLAS-ti™, render the study 
feasible. In addition, the fact that academic journals now publish in both digital and analogue 
formats, and have digitized past volumes, facilitates identification and retrieval through online 
journals and academic publication databases. The practical logistics of identifying and gathering 
a large body of scholarly literature are significantly easier to address since the advent of Google 
Scholar©. According to Harzing (2013), Google Scholar© has reached maturity, and is a 
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comprehensive, stable, reliable, un-biased source for citation information, particularly for recent 
social science publications. 

This partially completed project adds to the research synthesis knowledge base by 
demonstrating how emerging digital research tools and processes can be applied in innovative 
ways to bring order and insights to extensive bodies of literature. The successful pilot calibration 
and methodological refinement work completed so far has verified the feasibility of the full 
project, and it is anticipated that the completed research project and the online database will 
serve as exemplars for sharing open, online research literature resources. Although this particular 
study centers on the Col corpus of research, the methodology could be applied to any multi-vocal 
body of research. 
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Appendix A: Google Scholar Citations 

Col Chronological Google Scholar© Citations of Garrison, Anderson and Archer (2000) 
completed March 14, 2014. 


Garrison et al., 2000, Seminal 
Article* 

Anderson, et al. 
Teaching Presence... ** 

Garrison et al., 2001 
Critical thinking.. .*** 

Rourke et al.. 1999, 
Assessing Social 
Presence...**** 

Year 

Citation Count (Google Scholar© March 14, 2014) 

2001 

7 

2 

3 

7 

2002 

17 

21 

15 

36 

2003 

33 

43 

30 

50 

2004 

57 

45 

40 

59 

2005 

77 

62 

68 

92 

2006 

91 

78 

78 

83 

2007 

138 

95 

106 

128 

2008 

171 

107 

116 

115 

2009 

178 

109 

123 

143 

2010 

211 

114 

110 

128 

2011 

235 

107 

127 

125 

2012 

275 

128 

138 

112 

2013 

327 

134 

135 

142 

2014 

85 

30 

35 

30 

Total 

1902 

1075 

1124 

1250 


* Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer 
conferencing in higher education. The Internet and Higher Education, 2(2-3), 87-105. 

** Anderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing teaching presence in a computer 
conferencing context. Journal of Asynchronous Learning Networks, 5(2), 1-17. 

*** Garrison, D. R., Anderson, T., & Archer, W. (2001). Critical thinking, cognitive presence, and computer 
conferencing in distance education. American Journal of Distance Education, 75(1), 7-23. 

**** Rourke, L., Anderson, T., Garrison, D. R., & Archer, W. (1999). Assessing social presence in asynchronous 
text-based computer conferencing. The Journal of Distance Education / Revue de l ’Education A Distance, 
14(2), 50-71. 
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Appendix B: Proposed Study Document Characterization Pro-forma Tool 

Author(s)_ Date Published: 

Title: __ 


1. Bibliographic ID 
(in Zotero™) 

□ Journal article 

□ Journal 


□ Database 


□ Thesis/Dissertation: 
Masters 
Ed.D. 

Ph.D. 

Other_ 


2. Level of Evidence 

□ LoEl - trial or meta-anal. 

□ LoE2 - Lit Review or Synth 

□ LoE3 - multi-site 

□ LoE4 - single-site 

□ LoE5 - descriptive 

□ LoE6 - opinion 


3. Publication Status 

□ Published 

□ In press 

□ Unpublished 


4. Study Focus 

□ Curriculum design 

□ Development of methodology 

□ Ed. Tech 

□ Assessment of learning 

□ Exploration of relationships 

□ Instructional design 

□ Learning strategies 

□ Review of research 

□ Student retention 

□ Student support 

□ Teacher education 

□ Other_ 


5. Type of study 

□ Quantitative methodology 

□ Qualitative methodology 

□ Mixed-method 
QUANT-qual 
Quant-Qual 
quant- QUAL 

□ Meta-analysis or synthesis 

7. Data Collection Method (choose all that apply) 

□ Case study 

□ Survey 

□ Interview 

□ Observation 

□ Digital data collection (LMS) 

□ Other 

8. Educational setting (choose all that apply) 

□ Distance education 

□ Blended (define components) 

□ Fully online 

□ Face-to-face 

□ Flipped classroom 

□ Formal education (for credit) 

□ Informal education 

□ MOOC 

□ Private School 

□ Public School 

□ Other_ 

10. Location of study 

Country_ 

Province or state_ 

Other geographic descriptor_ 

Institution(s)/Agency(ies)_ 


6. Col Components 

□ Entire framework 

□ One presence (identify) TP SP CP 

□ Two presences (identify) 

□ All three presences 

□ Col concept (identify) 


□ Learning experience 

□ Other_ 


9. Study population 

□ K-12 public school students 

□ K-12 home school students 

□ Undergraduate students 

□ Graduate students 

□ Vocational students 

□ Adult upgrading students 

□ Pre-service Teachers 

□ Practicing Teachers 

□ Professional development participants 


11. Sample: n =_ 

□ Participants_ 

□ Course(s)_ 

□ Cohort(s)_ 

□ Department(s) 


12. Analysis 

□ Multivariate factor 
analysis 


□ ANOVA 

□ MANOVA 

□ Mean, Mode, Range 

□ Chi-square 

□ SD 


0: Analysis Cont’d 

□ Cronbach’s Alpha 

ifl[' Transcript/content Analysis 

□ Thick description 
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